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A  parallel  successive  overrelaxation  (SOR)  method  is  proposed  for  the  solution  of  the 
fundamental  symmetric  linear  complementarity  problem.  Convergence  is  established  under 
a  relaxation  factor  which  approaches  the  classical  value  of  2  for  a  loosely  coupled  problem. 
The  parallel  SOR -approach  is  then  applied  to  solve  the  symmetric  linear  complementarity 
problem  associated  with  the  least  norm  solution  of  a  linear  program. 
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SIGNIFICANCE  AND  EXPLANATION 


A  framework  for  the  parallel  solution  of  very  large  sparse  linear  programs  and  symmet¬ 
ric  linear  complementarity  problems  is  proposed.  Linear  programs  with  80,000  variables 
and  20,000  constraints  have  been  solved  with  the  serial  version  of  the  proposed  methods. 
Considerably  larger  problems  will  be  tackled  with  the  parallel  algorithms. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive  summary  lies 
with  MRC,  and  not  with  the  authors  of  this  report. 


PARALLEL  SUCCESSIVE  OVERRELAXATION  METHODS  FOR  SYMMETRIC 
LINEAR  COMPLEMENTARITY  PROBLEMS  AND  LINEAR  PROGRAMS 


O.  L.  Mangasarian1  and  R.  De  Leone2 


1.  Introduction 

The  purpose  of  this  work  is  to  propose  a  parallel  successive  overrelaxation  (SOR) 
method  for  the  solution  of  the  fundamental  symmetric  linear  complementarity  problem: 
Find  a  2  in  the  n-dimensional  real  space  Rn  such  that 

Mz  4-  q  >  0,  2  >  0,  z(Mz  -t  g)  =  0  (l) 

where  M  is  an  n  x  n  real  symmetric  matrix  and  q  is  a  vector  in  the  n-dimensional  real 
space  Rn  .  It  is  well  known  that  (1)  is  a  necessary  optimality  condition  for  the  optimization 
problem 

min  f{x):=  min  -zM z  -r  qz  (2) 

2>0  2>0  2 

and  that  (1)  is  a  sufficient  optimality  condition  for  (2)  if  M  is  positive  semidefinite.  Part 

of  the  importance  of  this  problem  stems  from  the  fact  that  the  solution  of  very  large 

sparse  linear  programs  by  successive  overrelaxation  methods  (Refs.  1-3)  can  be  reduced  to 
problem  (2)  with  a  positive  semidefinite  symmetric  matrix  M .  Until  recently  (Ref.  4)  only- 
serial  methods  had  been  proposed  for  the  solution  of  (1).  A  primary  result  of  this  work 
is  the  convergence  of  a  parallel  SOR  procedure  for  the  solution  of  (l)  under  a  relaxation 
stepsize  u.-  €  (0,  )  given  in  (15)  below,  where  a  is  a  positive  number  depending  on  the 

coupling  that  exists  between  the  row  blocks  of  the  matrix  M  that  are  being  coprocessed 
by  the  parallel  SOR  procedure.  If  the  row  blocks  are  loosely  coupled,  then  a  is  small.  In 
fact  o  is  zero  for  uncoupled  row  blocks  which  results  in  u;  6  (0.  2) ,  the  classical  interval 
for  the  serial  SOR  procedure  (Ref.  5).  We  give  now  a  brief  summary  of  the  paper. 
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We  begin  in  Section  2  with  some  known  preliminary  results  for  the  serial  SOR  algo¬ 
rithm  for  solving  (1).  A  key  role  will  be  played  by  Algorithm  2.1  below  which  is  a  special 
case  of  Algorithm  2.1  of  Ref.  1.  Although  Algorithm  2.1  was  originally  conceived  as  a 
serial  SOR  algorithm,  it  turns  out  that  it  is  general  enough  to  also  encompass  a  parallel 
SOR  scheme.  The  parallel  SOR  is  achieved  by  taking  a  decomposition  for  the  matrix  M 


other  than  the  standard  L  +  D  +  U  decomposition,  where  L  is  the  strictly  lower  triangular 
part.  D  is  the  diagonal  part  and  U  is  the  strictly  upper  triangular  part  of  M.  The  rest  of 
Section  2  of  the  paper  is  concerned  with  a  useful  modification  of  Algorithm  2.1  (Algorithm 
2.3)  and  with  convergence  results  for  both  algorithms.  In  Section  3,  which  constitutes  the 
core  of  the  paper,  we  construct  parallel  SOR  algorithms  for  the  solution  of  (l)  by  appropri¬ 
ately  decomposing  the  matrix  M  so  that  it  satisfies  the  assumptions  of  Algorithm  2.1  and 
such  that  parallel  computation  of  the  iterates  is  possible.  In  particular  we  decompose  the 
matrix  M  into  k  blocks  of  disjoint  row  matrices  and  apply  one  sweep  of  the  serial  SOR 
Algorithm  2.1  to  each  block  simultaneously.  After  this  sweep  the  information  is  shared 
among  the  k  blocks  and  the  process  is  repeated.  Under  an  appropriate  stepsize  condition 
we  show  in  Theorem  3.5  that  each  accumulation  point  of  such  a  parallel  SOR  algorithms 
solves  (l).  Theorem  3.6  establishes  convergence  of  all  the  iterates  of  parallel  SOR  algo¬ 
rithms  under  the  assumption  that  Mz  +  q  >  0  has  a  solution.  In  Section  4  we  apply  the 
parallel  SOR  algorithms  of  Section  3  to  finding  the  least  2-norm  solution  of  a  linear  pro¬ 
gram,  which  turns  out  to  be  precisely  a  problem  of  the  form  (2)  with  a  positive  semidefinite 
matrix.  Theorem  4.2  establishes  the  linear  convergence  of  parallel  SOR  schemes  for  the 
least  2-norm  solution  of  a  linear  program.  We  conclude  the  paper  in  Section  5  with  some 
brief  remarks  regarding  computational  implementation  of  the  proposed  parallel  schemes. 

We  briefly  describe  our  notation  now.  For  a  vector  x  in  the  n-dimensional  real  space 
Rn,  x+  will  denote  the  vector  in  Rn  with  components  (x+),  =  max  {zt,  0},  i  =  1  ,...,n. 
The  scalar  product  of  two  vectors  x  and  y  in  Rn  will  be  simply  denoted  by  xy.  For 

n 

1  <  p  <  oc  ,  the  p-norm  (  jit|p)1  p  of  a  vector  in  Rn  will  be  denoted  by  R”  will 

i  =  i 

denote  the  nonnegative  orthant  or  the  set  of  points  in  Rn  with  nonnegative  components, 
while  Rmxn  will  denote  the  set  of  all  mxn  real  matrices.  For  Ag  A T  will  denote 

the  transpose,  A,  will  denote  the  ith  row.  Al}  the  element  in  row  i  and  column  j,  and 
for  /  C  {l,...,m},  J  C  {l,...,n},  A/  will  denote  the  submatrix  of  A  with  rows  A,,  if/, 


-2- 


while  A jj  will  denote  the  submatrix  of  A  with  elements  AXJ ,  i  £/,;'£  J.  Similarly 
for  x  €  Rn  and  It  C  xj,  will  denote  xt.  i  e  It-  The  set  {/l5 12, /*}  is 

said  to  be  a  consecutive  partition  of  {l,...,n}  if  it  is  a  partition  of  such  that 

i  <  j  for  i  £  It,  j  6  and  L  =  -  1.  For  a  twice  differentiable  function 

B:Rm  x  Rn  — >  R,  y20  will  denote  the  (m  -1-  n)  x  (m  4-  n)  Hessian  matrix,  while  y0  will 
denote  the  rrn-  n  gradient  vector  and  V^(0,  0):  =  y(?(u,  u)|u=o,  v=o  ■  Here  and  throughout 
the  symbols  :  =  and  =:  denote  definition  of  the  term  on  the  left  and  right  sides  respectively. 
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% 

I 
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2.  Preliminary  Background 

In  this  section  we  give  some  background  results  needed  to  derive  our  parallel  SOR 
algorithm  for  solving  the  symmetric  linear  complementarity  problem  and  linear  programs. 
We  begin  first  with  a  special  case  of  the  serial  SOR  Algorithm  2.1  of  Ref.  1. 


Algorithm  2.1  (Serial  SOR  algorithm  for  (1))  Let  z°  >  0.  For  i  =  0,1,2,...,  let 


?‘+1  =  (z‘  -  uEt(Mzi  +  q  +  K'{z'+'  -  z‘))) 


where  u>  >  0,  and  {E1}  and  {K'}  are  bounded  sequences  of  matrices  in  J?nXn,  with  each 
E *  being  a  positive  diagonal  satisfying 


E'  >  al 


for  some  a  >  0  and  such  that  for  some  >  0 


y((u>2?‘)_1  +  K'  -  M/2)y  >  7||y||*  Vt,  Vy  €  Rn 


Remark  2.2  If  we  let 


L  +  D  +  U:  =  M 


where  L  is  the  strictly  lower  triangular  part  of  M ,  D  is  the  diagonal  of  M  and  U  is 
the  strictly  upper  triangular  part  of  M,  then  the  iteration  (3)  is  an  explicit  one  if  we  set 
K'  =  L,  U  or  0.  More  specifically  for  K'  =  L  or  U  and  El  —  D~l  (assuming  that  D  is 
positive)  iteration  (3)  gives  the  projected  SOR  algorithm  studied  in  Ref.  1  and  condition 
(5)  becomes  the  familiar  SOR  relaxation  factor  condition:  0  <  u>  <  2.  When  Kx  =  0  we 
have  the  projected  Jacobi  method  (Ref.  1).  However  K'  may  be  any  matrix  as  long  as 
(5)  is  satisfied,  in  which  case  the  iteration  (3)  can  be  considered  as  solving  a  (hopefully 
simpler)  linear  complementarity  problem. 

An  important  useful  modification  of  Algorithm  2.1  has  been  proposed  by  Subramanian 
(Ref.  6)  in  which  instead  of  taking  z,+  1  of  (3),  any  other  point  in  the  nonnegative  orthant 
is  taken  with  a  value  of  /  not  exceeding  f(z X*x).  This  leads  to  the  following  algorithm. 


Algorithm  2.3  (Modified  serial  SOR  algorithm  for  (1))  Let  z°  >  0.  For  i  =  0,1,2,..., 


s'  =  (zl  -  uE'iMz*  K'(s'  -  z*))) 


I 


WKWWi 


sy7i|.SV>t.v>1 


a 


where  w  >  0  and  the  sequences  of  matrices  {E1}  and  {K1}  satisfy  all  the  requirements 
of  Algorithm  2.1.  Choose  z,  +  1  >  0  such  that  f(z ,  +  1)  <  f(s'). 

The  important  point  to  note  about  Algorithm  2.3  is  that  is  allows  a  whole  class  of 
algorithms  to  be  based  on  Algorithm  2.1.  Typically,  2,_rl  of  Algorithm  2.3  is  obtained 
from  s'  by  some  sort  of  line  search. 

The  convergence  of  Algorithm  2.1  was  establsihed  in  Theorem  2.1  of  Ref.  1,  and 
the  convergence  of  Algorithm  2.3  in  Theorem  4.2  of  Chapter  3  of  Ref.  6.  We  combine 
these  results  into  the  following  fundamental  convergence  result  under  no  assumption  on 
the  matrix  M  other  than  symmetry. 

Theorem  2.4  (Convergence  of  serial  SOR  algorithms  for  (1))  Let  M  be  symmetric. 
Each  accumulation  point  of  the  sequence  {z1}  of  Algorithm  2.1  or  2.3  solves  the  linear 
complementarity  problem  (1). 

Note  that  Theorem  2.4  does  not  guarantee  the  existence  of  an  accumulation  point  for 
the  sequence  {z1} .  To  do  that  we  need  additional  assumptions  (Ref.  1,  Theorem  2.2)  such 
as  the  following. 

Theorem  2.5  (Strong  convergence  of  serial  SOR  algorithms  for  (1))  Let  M  be  symmetric 
and  positive  semidefinite,  and  let 

Mz  -+-  q  >  0  for  some  z  €  Rn  (8) 

Then,  the  sequences  {z1}  of  Algorithms  2.1  and  2.3  are  bounded  and  have  accumulation 
points.  Each  accumulation  point  of  {z1}  solves  (1). 

With  this  background  material  we  are  prepared  to  introduce  our  parallel  SOR  algo¬ 
rithm  for  solving  the  symmetric  linear  complementarity  problem  (1). 


3.  Parallel  SOR  for  the  Symmetric  Linear  Complementarity  Problem 

The  key  idea  of  our  approach  here  is  to  consider  K 1  of  Algorithm  (2.1)  as  a  sub¬ 
stitution  operator  which  replaces  the  old  data  zl  by  the  new  data  z'  +  l .  If  for  example 
K':  =  L,  where  L  is  the  strictly  lower  triangular  part  of  the  whole  matrix  M,  then  zj  +  I 
replaces  2)  during  the  computation  of  2^+1  for  ah  t  >  j.  Now,  consider  instead  the 
following  procedure.  Break  M  into  k  blocks  of  rows  as  follows: 


M  =: 


f  Af/,-1 

Mh 


(9) 


where  the  blocks  Mj3  correspond  to  the  variables  zj  and  {I\,  h,  h}  is  a  consecutive 
partition  of  {1,2,  ...,n}.  Now  partition  M/;  as  follows 


=:  [Mi3  i3 


M,  T  I 

h  h  J 


(10) 


where  is  the  complement  of  Ij  in  {1,2,  Thus  Mi]jj  is  a  principal  square  sub¬ 

matrix  of  M  with  elements  Mra,  r  6  Ij  and  s  6  Ij.  We  further  partition  M/./.  as 
follows 


=:  Lhh  +  Dhh  +  Uhh 


(11) 


where  L]jij  is  the  strictlly  lower  triangular  part  of  Djjij  its  diagonal  part  and 

Vj]j]  its  strictly  upper  triangular  part.  Thus  for  example  if  k  =  3  we  would  have  the 
following  decomposition  of  M 


Mjx 

Mhi, 

Mil  h  ^/,/3 

M  = 

M/, 

- 

M/,/, 

M]2}2  M  i2  7. 

M/, 

Mhi, 

Mj2i2  Mii2 

(12) 


Now  let  Kl  of  Algorithm  2.1  be  defined  by  a  block  diagonal  matrix  as  follows 


r 


Kl  =  K : 


'h h  -» 


(13) 


where  each  L/j/]  is  a  strictly  lower  triangular  matarix  defined  in  (11). 


Algorithm  2.1  can  now  be  performed  for  each  row  block  I3,  j  =  l,...,fc,  simultane¬ 
ously,  that  is  in  parallel.  Note  that  this  is  not  a  block  Jacobi  iteration.  More  specifically 
we  have  the  following  algorithm. 

Algorithm  3.1  (Parallel  SOR  for  (1))  Let  {li.  J2,  ■■■,  h}  he  a  consecutive  partition 
of  {l,2,...n},  let  the  diagonal  D  of  M  have  positive  elements  and  let  z°  >  0.  For 
i  =  0, 1,2, ...,  let 

*;;■  =  H  -  -  *;,») 


3  =  1  ,-,k 


where 


0  <  lj  <  mm  mm  - ^ -  (15) 

i <j<k  eei,  i  1  -  n 

1  +  3^j  Du 

Remark  3.2  Iteration  (14)  can  be  performed  in  parallel  on  k  processors.  The  new  value 
z,+1  must  then  be  shared  between  the  k  processors. 

Remark  3.3  If  all  MI}jj  are  zero  then  0  <  <  2  for  all  j ,  which  is  the  standard 

SOR  relaxation  factor  range.  This  corresponds  to  k  uncoupled  linear  complementarity 
problems.  If  all  Mj  j  are  small  relative  to  Dj},  which  corresponds  to  a  loosely  coupled 
linear  complementarity  problem,  the  upper  bound  on  given  by  (15)  is  close  to  2. 

We  state  now  a  parallel  SOR  version  of  Algorithm  2.3. 

Algorithm  3.4  (Modified  parallel  SOR  for  (l))  Let  the  assumptions  of  Algorithm  3.1 
hold.  For  >  =  0,1,2,...,  let 

*'h  =  (2L  ~  uDij />  [Mhz'  +  “  2/7)))+ 

j  =  1, 

where  w  satisfies  (15).  Chose  z,  +  1  >  0  such  that  /(z,J_1)  <  /(s'). 

We  can  establish  the  convergence  of  Algorithms  3.1  and  3.4  by  appealing  to  Theorem 
2.4.  We  have  then  the  following  convergence  result. 

Theorem  3.5  (Convergence  of  parallel  SOR  algorithms  for  (1))  Let  M  be  symmetric. 
Each  accumulation  point  of  the  sequence  {z'}  of  Algorithms  3.1  and  3.4  solves  the  linear 
complementarity  problem  (1). 


Proof  By  Theorem  2.4  we  only  need  to  establish  that  condition  (5)  is  satisfied  by  the 
choice  (13)  for  K\  E':  =  D~l  and  w  satisfying  assumption  (15).  We  have 


+Kl  -  M/2)y 
=  ^y(2w_1£>  4-  2 K  -  M)y 
1  k 

=  2  Y'.yir(2u>  l Dir  +  2 Kit  -  M/Jy 
r  =  1 

= 

r=l 
1  k 

=iS>. 

r  =  1 
1  * 

-;5> 


(2u»  1  D/rjt  +  2K irjr  -  Mjrir)yir  -  Y'Mi'J,  yj , 

s^r 


(2w  1  Dir ir  +  2L/,/,  -  {Ljrjr  +  D/r/r  +  UIrir))yiT  -  ^ M/r/ ,  y/, 

s#r 


r  =  l 


(2w  1  -  l)£/r/,y/r  -  y;„ 

(Because  L  —  UT) 


i 


r  =  J 


(2w  1  -  !)£>/,/,  -  M/r/r 


1 

Vir. 

1  -  1 

=  o  i  y] 


(2u;-‘  -1)0/.  fl 


-M 


/*  /* 


(2a;-1  -  1)Z>/*/*  J 


>  nr  iyli! 


where  last  inequalilty  holds  for  some  >  0  because  of  the  positive  definiteness  of  the 
symmetric  n  x  n  matrix  (preceding  the  inequality)  which  is  induced  by  its  row  diagonal 
dominance  (Ref.  5).  The  row  diagonal  dominance  is  precisely  a  consequence  of  assumption 
(15).  I 

Having  established  that  the  parallel  SOR  Algorithms  3.1  and  3.4  can  be  considered  as 
special  cases  of  the  general  serial  Algorithms  2.1  and  2.3  respectively,  the  following  strong 
convergence  result  is  a  direct  consequence  of  Theorem  2.5. 


Theorem  3.6  (Strong  convergence  of  parallel  SOR  algorithms  for  (1))  Let  M  be  sym¬ 
metric  and  positive  semidefinite  and  let  assumption  (8)  hold.  Then  the  sequences  {2’}  of 


Algorithms  3.1  and  3.4  are  bounded  and  have  accumulation  points.  Each  accumulation 
point  of  { z *}  solves  (1). 

Remark  3.7  Minor  changes  in  the  proof  of  Theorem  3.5  allows  us  to  have  a  different 
for  each  j  =  1  in  (14).  In  particular  all  we  need  is  that  for  j  =  wy  must 

satisfy 

2 

0  <  wy  <  min  - ^ -  (17) 

£<E/j  1  +  ~j  \Mis\/Dtt 

This  results  in  larger  stepsizes  for  Algorithms  3.1  and  3.4. 

We  now  turn  our  attention  to  the  parallel  solution  of  linear  programs. 


4.  Parallel  Solution  of  Linear  Programs 

The  key  idea  here  is  to  find  the  least  2-norm  solution  of  a  linear  program  by  converting 
the  problem  to  a  positive  semidefinite  linear  complementarity  problem  (Refs.  2-3)  and  to 
use  the  parallel  SOR  procedures  proposed  in  the  previous  section.  We  will  present  a 
parallel  implementation  here  of  the  linearlly  convergent  iterative  scheme  proposed  in  Ref. 
7. 

We  consider  the  linear  program 

min  cx  subject  to  Ax  >  b,  x  >  0  (18) 

X 

where  c  €  Rh,  b  €  Rm  and  A  €  Rmxh ,  and  its  dual 

max  bu  subject  to  ATu  <  c,  u  >  0  (19) 

U 

It  is  known  (Refs.  2-3)  that  x  is  the  unique  least  2-norm  solution  to  (18)  if  and  only  if  x 
is  the  unique  solution  to  the  quadratic  program 

min  cx  +  |  xx  subject  to  Ax  >  b,  x  >  0  (20) 

x  2 

for  all  e  €  (0,  e]  for  some  e  >  0.  The  dual  to  the  quadratic  program  (20)  is 

max  -  -  4-  bu  subject  to  t;  =  ex  —  ATu  +  c,  (u,v)  >  0  (21) 

x,u, v  2 

To  solve  (20)  for  a  fixed  positive  e  we  shall  use  the  parallel  SOR  procedures  of  Section  3 
applied  to  its  dual  (21)  with  the  variable  x  eliminated  through  the  dual  constraint 

x  =  (ATu  +  v  -  c)/e  (22) 

and  thus  obtaining  the  dual  problem 

min  0(u,v):=  min  -  \\Atu  -+•  v  -  c j; ^  -  efcu  (23) 

(u,v)>0  2 

which  is  precisely  of  the  form  (2)  with  a  positive  semidefinite  matrix  and  hence  is  equivalent 
to  the  symmetric  linear  complementarity  problem  (1)  on  RmJth  with  M:=  V2^(«.*;) 
and  q:=  y0(O,O).  We  shall  now  describe  a  linearly  convergent  sequential  parallel  SOR 


procedure  for  solving  (23)  based  on  Ref.  7  and  the  results  of  Section  3.  We  first  need  a 
definition. 


Definition  4.1  (Approximate  solutions  to  (23)  and  (20))  For  a  fixed  positive  e  any  point 
in  R+  ~  is  an  approximate  solution  to  the  dual  quadratic  program  (23)  and  is  designated 
by  (u(f),  v(e)).  The  corresponding  x(e)  in  Rh  defined  by  (22)  with  (u,v)  =  (u(s),  v(s)) 
is  an  approximate  solution  to  the  quadratic  program  (20).  The  residual  r(e)  associated 
with  ( u(e ),  v(e),  i(e))  is  defined  by 


r(£r);=  [|x(e)r(e)  +u(e)(Ax(e)  -  6)j  +  || (6  —  Ax(<r))J| 

+  ii(  ~  X(e))  +  lloJ 


OO 

1/2 


(24) 


Note  that  for  an  e  >  0  and  an  approximate  solution  (u(£-),  v(e))  to  (23)  and  a 
corresponding  approximate  solution  x(e)  to  (20),  r(e)  —  0  if  and  only  if  (u(e),  v(e))  is 
an  exact  solution  of  (23)  and  x(e)  is  the  unique  exact  solution  of  (20). 

We  are  prepared  now  to  state  and  prove  a  linearly  convergent  parallel  SOR  procedure 
for  computing  the  least  2-norm  solution  of  the  linear  program  (18). 

Theorem  4.2  (Linearly  convergent  parallel  SOR  for  linear  programs)  Assume  that  the 
linear  program  (18)  is  solvable  and  that  6^0.  Let  {eo,  £i,...}  be  a  decreasing  sequence 
of  positive  numbers  such  that 


e,  +  1  =  nei  for  some  n  6  (0,1)  (25) 

and  let  {u(e») ,  u(£t),  x(e,)}  be  a  corresponding  sequence  of  approximate  solutions  to  (23) 
and  (20)  satisfying  Definition  4.1  and  obtained  by  either  of  the  parallel  SOR  Algorithms 
3.1  or  3.4  applied  to  (23)  and  such  that  their  residuals  as  defined  by  (24)  satisfy 

'•(e.  +  i)  <  l/r(ert)  (26) 

for  some  u  >  0  such  that 

(27) 

Then  the  sequence  {x(e,)}  converges  to  x ,  the  least  2-norm  solution  of  the  linear  program 
(18)  at  the  linear  root  rate 


-11- 


(28) 


||x(e,)  -  *||s  <  6(vln'r‘Y  for  i  >  i 
for  some  constant  6  and  some  integer  i . 

Proof  See  Theorem  3.7  of  Ref.  7.  I 

5.  Conclusion 

We  have  presented  a  framework  for  the  parallel  solution  of  symmetric  linear  comple¬ 
mentarity  problems  and  linear  programs.  The  proposed  SOR  algorithm  is  best  suited  for  a 
tightly-coupled  shared-memory  multiprocessor  such  as  the  one  to  be  acquired  by  the  Com¬ 
puter  Sciences  Department  at  Madison.  However  we  plan  to  test  the  proposed  algorithm 
on  the  existing  loosely-coupled  20-processor  token-ring-connected  Crystal  machine  (Ref. 
8)  of  the  Computer  Sciences  Department  in  order  to  develop  efficient  computational  im¬ 
plementations  of  our  algorithm.  Because  we  have  been  able  to  solve  sparse  linear  programs 
of  size  20,000  variables  and  5,000  constraints  by  the  serial  version  of  our  SOR  procedure 
in  78  minutes  on  a  VAX  11/780  (Ref.  2),  we  are  hopeful  of  solving  substantially  larger 
problems  by  our  parallel  approach. 
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